Program transformation landscapes for automated program modification using Gin

Abstract Automated program modification underlies two successful research areas — genetic improvement and repair. Under the generate-and-validate strategy, automated transforms a program, then validates result against test suite. Much work has focused on search space of application single fine-grained operators copy , delete replace swap at both line statement granularity. This explores limits this strategy. We scale up existing findings an order magnitude from small corpora to 10 real-world Java programs comprising 500k LoC. decisively show that grammar-specificity granular edits pays off: its pass rate triples uses 10% less computational resources. confirm previous is most effective operator for creating test-suite equivalent variants. go farther than prior by exploring ’s effectiveness exhaustively applying it. strategy too costly in practice be used improved software further find rates drop 12–34% 2–6% 5-edit sequences, which implies progress will need human-inspired target specific faults or improvements. A amenable extent automatically editing it likely produce passing are first systematically code measure correlates with program’s amenability modification. found no strong correlations, leaving question open.